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ABSTRACT 


In  this  paper  approximate  models  are  given  to  describe 
backlogs  of  demands  at  a  service  system  confronted  by  finitely 
many  users.   An  application  is  to  a  time-shared  computer  center. 
Various  functionals  of  the  number  waiting  are  discussed  using 
the  approximation. 


DELAYS  AT  A  FACILITY  WITH  DEMAND 

FROM  MANY  DISTINCT  SOURCES 

D.  P.  Gaver 
Naval  Postgraduate  School 
Monterey,  California 

1.   Introduction. 

Many  computation  centers  and  other  job-shop-like  systems,  e.g. 
repair  shops,  are  characterized  by  time-dependent  random  demand,  and 
by  sometimes  considerable  work  backlogs.   Computation  centers  that 
service  terminals  also  have  the  peculiarity  that  if  all  customers  are 
currently  waiting  in  queue  for  attention,  or  are  receiving  service  in 
a  round-robin  fashion,  then  no  further  demands  can  take  place.   This 
feature  is  characteristic  of  many  "repairman"  type  problems,  the  latter 
being  prototypes  of  numerous  cyclic  queueing  situations;  see  Feller  [4] 
for  a  classical  treatment,  and  Gordon  and  Newell  [6]  for  other  develop- 
ments . 

In  this  paper  several  new  approximate  models  are  given  for  the 
computer  center  problem.   The  models  explicitly  confront  the  time- 
dependent  demand  problem,  where  the  latter  refers  to  daily  demand  if 
desired.   The  results  are  easy  to  understand  and  compute  with;  they 
apply  most  accurately  when  the  number  of  system  users  is  large,  as  will 
often  be  true. 


2 .   Assumptions . 

Let  m   terminals  be  connected  to  a  main  computer  facility.   At 
time   t   suppose  that  the  rate  of  new  demand  from  each  terminal  is 
A(t).   Effectively  we  are  assuming  that  during  hour   t   the  tota.l 
number  of  demands  is  a  pure  birth  process  (see  Feller  [4]),  or  approxi- 
mately Poisson  for  short  time  intervals  (non-time-homogeneous) ,  provided 
the  terminals  are  occupied  and  their  users  are  not  awaiting  computational 
return  (service)  from  the  facility. 

Assume  that  the  rate  of  completion  of  active  user  interactions 
is  a  constant,   u,   so  that  if  a  number  of  users  are  awaiting  or  under- 
going computational  attention  at  time   t,   the  chance  that  one  leaves 
is   udt   to  terms  of  order   dt.   This  assumption  is  in  keeping  with  a 
first-come,  first-served  discipline;  more  importantly,  it  conforms 
approximately  to  a  round-robin  discipline,  in  \\?hich  each  customer  present 
receives  a  small  quantum  of  computer  time. 


3.   Model  Equations. 

The  above  assumptions  imply  that   Q(t)  ,   the  number  of  terminal 
users  that  await  or  undergo  service  at  time   t,   is  described  by  a 
birth  and  death  stochastic  process,  see  Feller  [4].   One  can  envision — 
but  not  relish — the  task  of  writing  down  m  +  1   simultaneous  differential 
equations  for  the  probabilities  that   Q(t)  =  j   (j  =  0,1,..., m),   working 
out  a  solution,  and  then  interpreting  the  results. 

The  solution  must  necessarily  be  numerical,  and  will  be  tedious 
and  expensive  to  obtain  if   m  grows  large.   We  present  here  an  approach 
that  is  approximate  but  should  be  quite  accurate  when  m   is  moderately 
sizable,  and  business  is  brisk  at  the  computer.   The  approach  is  based 
on  approximating  Q(t)   by  a  deterministic  part,  plus  an  additional 
noise,  the  latter  being  characterized  as  a  random  process.   In  the 
present  situation  the  noise  turns  out  to  be  a  familiar  Gaussian  diffusion 
process,  the  Ornstein-Uhlenbeck,  see  Cox  and  Miller  [1].   It  has  proper- 
ties that  may  be  exploited  in  order  to  shed  light  on  the  computer's 
service  capabilities. 

If   m,   the  number  of  terminals,  is  large  then  it  is  plausible 

that  -* -»■  q(t)  ,   a  deterministic  function.   That  the  latter  is  true 

m 

for  constant   X(t),   equal  to   X,   follows  from  results  of  Iglehart, 
for  the  closely  related  repairman  problems;  see  [7].   Next  consider 

N  (t)  =  Q(t)  "  mq(t)  (3.1) 

mv  '  /— 

vm 

where  the  latter  is  the  "noise"  process  for  finite  m.   We  shall  study 

N  (t)   as   m  ■+  «>,   and  then  approximate  Q(t)   by 


Q(t)  «  mq(t)  +  v^  N(t) ,  (3.2) 


N(t)   being  the  limiting  noise.   Note  that  if   m   is  made  large  our 
assumptions  imply  that  demand  rate,  and  hence  queue  size  grow  indefi- 
nitely.  To  obtain  a  sensible  limit  we  must  scale:   my   is  now  the 
departure  or  interaction  completion,  rate. 

Argue  as  follows  concerning  changes  in  Q(t)   in  a  short  time 
interval  of  length   dt :   Q(t)   moves  deterministically  by  an  amount 
{ [m-Q(t) ]X (t)-my }dt   except  when  Q(t)  =  0;  furthermore,  the  variance 
of  the  displacement  is  sxmply    {  [m~Q  (t)  ]  X  (t)-hny  }dt  4-  o(dt).   Assuming 
that  we  wish  to  approximate  by  a  diffusion,  write  the  stochastic 
differential  equation 


dQ(t)  =  {[m-Q(t)]X(t)-mu}dt  +  /{  [m-Q(t)  ]+my}  dW(t)       (3.3) 

where  dW(t)   is  the  "differential"  of  a  Wiener  process  (distributed 
like  a  normal  random  variable  with  mean  zero  and  variance   dt) .   Doob 
[3]  addresses  the  mathematical  problems  that  arise;  also  Cox  and  Miller 
[1] ,  Gikhman  and  Skorokhod  [5],  and  McKean  [8], 

Next  apply  the  normalization  (3.1).   We  have 


dQ(t)  =  /m~dN  (t)  +  mdq(t)  = 
m 

{[m(l-q(t))-v/mNm(t)]X(t)-my}dt 

+  /{[m(l-q(t))-^N  (t)]X(t)+my}  dW(t)    (3.4) 


m 


After  division  by  m  we  obtain 

dN  (t)  +  Jn   dq(t)  =  {  [  »/m(l-q(t) )  -  N  (t)]X(t)  --  Jm   y}dt 
m  m 


+  /   (l-q(t))  - 


m 


X(t)  +  y  dW(t)    (3.5) 


Clearly  if   m  ■+■  °°   the  above  equation  makes  sense  only  if  coefficients 
of  /iii  cancel  out;  this  leads  to  a  differential  equation  for  the  deter- 
ministic part,   q(t)  : 

j*   =  \(t)[l-q(t)]  -  u.  (3.6) 

Taking  m  ->  °°   in  the  remaining  equation  yields  the  stochastic  differ- 
ential equation 

dN(t)  =  -A(t)N(t)  +  /[(l-q(t))X(t)+u]  dW(t) .         (3.7) 

The  latter  is  recognized  as  describing  a  non-stationary  Ornstein- 
Uhlenbeck  diffusion  process;  see  Cox  and  Miller  [1],  p.  225.   We  have 
neglected  the  boundary  condition  at  zero  (no  queue  can  become  negative) 
and  hence  our  solution  will  be  valid  only  when  zero  is  seldom  visited, 
i.e.  when  A(t)  >>  u ,   at  least  most  of  the  time.   But  this  is  precisely 
the  peak  load  problem  that  is  often  of  interest  to  designers  and  users. 

Deterministic  Component:   Solution 

The  equation  (3.6)  may  be  solved  by  routine  methods.   We  find 


-/h(s)ds    ft  -/tX(s)ds 

q(t)  =  q(0)e   U       + 


Jo 


[X(x)-u]e   X       dx      (3.8) 


In  case  X (x)  <  y   the  relevant  portion  of  the  integrand  should,  in  the 
spirit  of  our  approximation,  be  replaced  by  zero. 
Example:   X(t)  =  X,   a  constant. 
In  this  case,  for   X  >  u 

-At    r„    u\   ,„   -Xt 


q(t)  =  q(0)e  At  +  (l  -^)(l-e~At)  (3.9) 


and  in  the  long  run   (t-*») 


,(t)-l-* 


(3.10) 


A  little  thought  reveals  the  plausibility  of  this  result.   Our  system  is 

behaving  like  an  infinite  server  system  fed  by  a  Poisson  source,  where 

the  servers  are  now  the  terminals,  and  the  computer  is  the  source  (of 

returned  transactions) .   The  expected  number  residing  at  the  terminals 

is  approximately  m  — ,   the  remainder  waiting  or  being  served  at  the 

A 


computer, 


Stochastic  Component:   Solution 

A  formal  solution  of  (3,7)  is  seen  to  be 


N(t)  =  N(0)e 


-/Jx(s)ds    rt   -/*X(s)d 
+    e 


a(x)dW(x) 


(3.11) 


where 


a(t)  =  /[(l-q(t))A(t)+u] 


(3.16) 


and  the  integral  involving  dW(*)  may  be  interpreted  by  integration  by 
parts.  Apparently  N(t)  is  normal  or  Gaussian  with  expectation  N(0)e 
and,  given  N(0),   we  find 


-JJix(s)dj 


E[(N(t)-N(0)e 


-fh(s)ds      rt   -2/tA(s)ds 


)z] 


a^(x)dx 


(3.13) 


Var[N(t)] 


Example:   X(t)  =  X. 

In  this  case  (3.13)  becomes 


where 


E[(N(t)-N(0)e  At)2]  = 


■2X(t-x)   2 

a^(x)dx, 


0 


(3.14) 


-At 


a2(t)  =  y(2-e  At)  +  [l-q(0)]Xe 


-Xt 


(3.15) 


thus 


E[(N(t)-N(0)e  Xt)2]  -  f{l-e   2Xt)  , 


=  Var[N(t)]. 


(3.16) 


There  is  close  agreement  between  the  long-run   (t-*°°)   moments  of  Q(t)  , 
and  of  the  approximation  (3.2)  when  X  >>  u . 


4 .   Applications  of  the  Approximation. 

The  mathematical  results  obtained  may  be  useful  in  assessing 
various  aspects  of  system  performance.   We  briefly  describe  these. 

a)  Distribution  of   Q(t),   the  number  of  terminal  customers  waiting. 

In  our  approximation,   Q(t)   is  normally  distributed  such  that 

E[Q(t)]  *  mq(t),  (4.1) 

with   q(t)   given  by  (3.8),  and   Var[N(t)]   given  by  substituting  into 
(3.13).   One  should  find  such  approximations  useful  for  quick  estimates 
of  time-dependent  behavior,  and  also  for  checks  of  simulation  validity. 

b)  Distribution  of  total  waiting  time. 

It  may  be  of  interest  to  describe  the  total  accumulated  waiting 
time  of  all  applications  for  service  during  a  fixed  time  period,  e.g. 
one  day  ; 


Q(t)  = 
Our  approximation  amounts  to  writing 


Q(r)dr 
0 


Q(t)  =     (mq(r)  +  v^  N(r)  }dr 
in 


which  immediately  implies  that   Q(t)   is  approximately  normally  distrib- 
uted with  parameters 


E[Q(t)]  -  m 
and,  if   N(0)  =  0, 


t 

q(r)dr 
0 


N(r)dr]  (4.2) 

0 


Var[Q(t)]  =  m  Var[ 
v7here  the  latter  may  be  simplified  by  integration  by  parts. 


Example:   A(t)  =  A. 

Evaluation  of  (4.2)  requires  an  expression  for  the  variance  of 
the  integral  of   N(r).   Integration  by  parts  gives 


N(t)  = 


N(r)dr  = 
0  ^O 


ri_e-A(t-xh 


o(x)dW(x) 


(4.4) 


and  thus 


Var[N(t)]  = 


ri_e-A(t-x)n 


a2(x)dx 


(4.6) 


If   q(0)  -  N(0)  =  0,   then  by  (3.15) 


Var[N(t)]  = 


l-e~X(t~x)12 


0L 


[2u+(A-y)e  Ax]dx 


2y 


X 


as   t  -»- 


(4.7) 


Perhaps  of  more  interest  in  the  context  of  computer  system  applications, 
where  interest  focuses  on  t   of  the  order  of  hours,  and  X  on  a 

one-minute  scale,  the  above  is  expressible  as 


Var[N(t)]  ~|£  t 


(4.8) 


as  X   ->  °°  for  fixed   t. 

c)   Probabilistic  bounds  on  Q(t) . 

Although  we  have  found  in  a)  an  approximate  normal  distribution 
for   Q(t)   at  any  fixed  time   t,   it  would  be  desirable  to  provide  infor- 
mation on  Q(t)   for  all   t   during  a  particular  period.   For  example, 


10 


we  would  like  to  be  able  to  state  that,  during  a  day,  queue  size  ever 
exceeds  some  specified  level  only  with  a  given  (small)  probability. 
Such  statements  can  be  made  if  we  can  derive  a  boundary,   8(t) ,   such 
that 

P{Q(t)  <£  8(t),   all   t  s£  T}.  (4.9) 

is  specified. 

In  fact,  such  boundaries  may  be  derived  mathematically  and  we 
describe  two  of  them.   Unfortunately  these  are  not  entirely  appealing 
from  a  practical  viewpoint:   either  the  boundary  shape  is  unsatisfactory, 
or  the  probability  associated  with  a  boundary  crossing  is  not  expressed 
in  a  usable  mathematical  form. 
c-1)   Transformation  to  Wiener  process  with  linear  boundary. 

Our  first  boundary  problem  is  derived  by  expressing  N(t) ,   the 
noise  process,  as  a  Wiener  process,  and  referring  the  latter  to  a  linear 
boundary.   Probabilities  for  crossing  such  boundaries  are  well  known. 

Transform  (3.7)  into 

J^A(s)ds     Jh(s)  

d[N(t)e  U      ]=eU    /[(l-q(t))A(t)+y]  dW(t)  ,      (4.10) 

J*QA(s)ds 
from  which  it  is  clear  that  the  process   {N(t)e        }  is  Wiener  with 

2/^X(s)ds 
zero  mean  and  infinitesimal  variance  equal  to   e         [ (l-q(t) ) X (t)+u J 

A  time-scale  transformation  T(t)   converts  the  Wiener  process  into  one 

with  stationary  increments;  necessarily 

2/^X(s)ds 
dx  =  e         [(l-q(t))A(t)+u]dt  (4.11) 


or 
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T(t)  = 


t   2/^X(s)ds 


[(l-q(r))X(r)+u]dr. 


(A. 12) 


0 


Thus 


N(t)e 


lh(s)di 


=   V(x), 


(4.13) 


a  zero-drift  Wiener  process  with  unit  infinitesimal  variance  and 
V(0)  =  N(0)  =0.   It  is  known  (see  Cox  and  Miller,  pp.  220-221)  that 
if  a  >  0 


P{V(x)  £  a  +  b  ,  ¥  x  <;  x}  -  *(^kl)  _  e"2ab  $  (^j±l)  ?    (4>14) 


0  being  the  standard  normal  distribution  function. 
Thus  with  the  above  probability,  (4.11), 


/F 


N(t)  £  {a  +  b 


t   2/^X(s)ds 


[(l-q))X(r)+u]dr}e 


-/Jx(s)ds 


(t)   (4.15) 


0 


for  all   t,0  ^  t  £  T,   and   x(t)  =  t.   From  this  result  we  can  state 
that  with  (approximate)  probability  (4.14) 


Q(t)  £  mq(t)  +  •m  B(t)  . 


(4.16) 


Example.   X(t)  =  X,   a  constant. 
Let   q(0)  -  0.   Then 


B(t)  =  {a  +  b 


e2Xr[2y+(X-Vi)e  Xr]dr}e  Xt 


0 


=  ae    +  b[(l  -  -r)  +  f-  e   -  e        (4.17) 

A        A 


and  hence 
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Q(t)  £  m[(l  -  y)(l-e  Xt)]  +  •£  B(t)  (4.18) 


for  all   t  £  T; 


T  = 


T 

e2Ar[2y+(X-y)e"Xr]dr  -  ^(e2AT-l)  +  (1  -  £)  (eXT-l)    (4.19) 
0  A  A 


so  for  given  T  one  solves  for   x   required  to  evaluate  (4.18). 
Unfortunately,  the  above  boundary  tends  to  be  excessively  U-shaped;  in 
particular  it  is  very  high — and  hence  uninf ormative — for  large  T. 
c-2)   Straight  line  boundary  for  stationary  Ornstein-Uhlenbeck  noise. 

Let  us  consider  the  special  case  of   A(t)  =  A   a  constant.   Then 
N(t) ,   the  limiting  noise,  eventually  becomes  Ornstein-Uhlenbeck,  satis- 
fying 

dN(t)  -  -AN(t)dt  +  /2~v7  dW(t)  (4.20) 

or 

dN(t')  =  -AN(t')dt'  +  dW(t')  (4.21) 

if   t'  =  2ut.   The  time  to  reach  a  boundary  8(t)  =  c  >  0   from  x, 

denoted  by  T   has  a  Laplace  transform  that  may  be  expressed  in  terms 

of  Weber,  or  parabolic  cylinder,  functions;  according  to  Darling  and 

Siegert  [2] 

-sT  D_  (x) 

E[e   °|N(0)  =   x)    =   ^-^y  ,  (4.22) 

-s 

but  the  actual  probability  distribution  of   T    is  not  given  numerically 
in  the  paper  cited,  and  I  have  not  seen  a  numerical  tabulation  in  the  litera- 
ture.  Actually,  such  a  tabulation  would  be  useful  for  one  could  then  assess 
the  probability  distribution  of  max  Q(t)   over  a  one-day  period. 
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5.   Concluding  Remarks. 

The  above  model  may  be  generalized  in  various  directions,  some 
of  which  are  likely  to  be  useful. 

a)  Service  rate,   u,   need  not  be  a  constant,  but  may  be  made  a  function 
of  time.   In  particular,  justification  of  the  time-dependent  results 

for  arrival  and  service  functions  that  are  constant  over   t-intervals 
should  be  straightforward.   A  time-dependent  service  rate  might  be 
required  to  reflect  the  difference  in  business  submitted  (transaction 
type  changes)  throughout  a  day.   Time  dependent  changes  in  m,   the 
number  of  active  terminals,  may  also  be  handled  on  a  piecewise  constant 
basis . 

b)  Service  need  not  be  exponential-Markovian.  If  traffic  is  heavy  and 
transactions  require  arbitrarily  independently  distributed  random  times 
S   to  terminate,  then  it  is  natural  to  substitute  the  renewal  theoretic 

asymptotic  variance  into  the  stochastic  differential  equation  (3.7): 

-3 
simply  replace  u  by  Var[S](E[S])    under  the  radical. 

Finally,  it  is  of  interest  that  a  variety  of  different,  but 

similar,  stochastic  models  for  cyclic  service  result  in  the  same  limiting 

"noise"  process  to  be  superimposed  upon  the  deterministic  solution.   This 

sort  of  robustness  encourages  one  to  think  that  simple  models  of  the  type 

suggested  will  often  work  adequately  to  generate  useful  insights,  and 

to  supplement  or  replace  more  complex  calculations  or  simulations. 
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